Partial Generalized Additive Models: An Information-Theoretic Approach for Dealing With Concurvity and Selecting Variables

Scientists are often interested in which covariates are important, and how these covariates affect the response variable, rather than just making predictions. This requires inputs from both statistical modeling and background knowledge. Generalized additive models (GAMs) are a class of interpretable, multivariate nonparametric regression models which are very useful data exploration tools for these purposes, but concurvity among covariates (the nonlinear analogue of collinearity for linear regression) can lead GAMs to produce unstable or even wrong estimates of the covariates’ functional effects. We develop a new procedure called partial generalized additive models (pGAM), based on mutual information (MI), a measure of nonlinear dependence between variables. Our procedure is similar in spirit to the Gram–Schmidt method for linear least squares. By building a GAM on a selected set of transformed variables, pGAM produces more stable models, selects variables parsimoniously, and provides insight into the nature of concurvity between the covariates by calculating functional dependencies among them. With simulation experiments and real-data examples, we show that pGAM produces much better estimates of the covariates’ functional effects, and also incorporates a reasonable and meaningful variable selection method. R code for fitting pGAMs is available online (see Supplemental Materials Section).